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SYSTEM AND METHOD FOR DEVELOPING CRITICAL THINKING 

CROSS-REFERENCE TO RELATED APPLICATIONS 
[0001] This application claims priority pursuant to 35 U.S.C. Section 119 from United 

States Provisional Application No. 60/392,563 filed June 28, 2002. 

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR 

DEVELOPMENT 

[0002] None. 

BACKGROUND OF THE INVENTION 
[0003] The present invention provides a computer-based method of and system for 

teaching thinking skills. More particularly, the present invention provides a method and system 
in which a learner is taught to think critically about content presented in a computing 
environment. 

[0004] Increasingly, teachers and others are using individual and networked computers to 

present information to students and other learners and to question, quiz or test the learners about 
the information. Unfortunately, the present systems and methods do not seek to develop critical 
thinking skills in addition to or in conjunction with teaching a particular subject matter. Instead, 
the present systems and methods are satisfied with rote memorization. 

[0005] Critical thinking skills have been said to include interpretation, analysis, 

evaluation, inference, explanation, and self-regulation. Clearly, these skills go beyond merely 
viewing and then memorizing information. More specifically, critical thinking makes it possible 
for a learner to do more than merely answer a question; critical thinking allows a learner to 
provide some rationale or justification for his or her answer. Thus, to teach a learner to think 
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critically, a system and method of teaching must require that the learner provide some 
justification for his or her answer and then provide information so that the learner may critique 
his or her answers and justifications. The present invention accomplishes this and other 
objectives. 

BRIEF SUMMARY OF THE INVENTION 

[0006] The present invention encompasses a computer-based system and method for 

teaching critical thinking skills. In one embodiment, the present invention is a method in which 
information and a question are presented to a learner and the learner is asked to answer the 
question and justify his or her answer. The learner may view reference material to assist in 
answering the question and justifying his or her answer. After providing an answer and a 
justification, a statistical analysis is performed. Thereafter, a number of results are provided to 
the learner. These results may include not only the outcome of the analysis, but also other 
learners' answers and justifications. The learner may then review an expert's opinion regarding 
an acceptable answer and justification. Also, after receiving the results, the learner may interact 
with other learners' answers and justifications, such as by selecting the best justifications, rating 
the justifications, and e-mailing comments to other learners. 

[0007] In another embodiment, the present invention is a computer-based system that 

includes modules that are operable to receive and store informational content and questions and 

to provide the information and questions to learners. The system also includes a coach module, a 

resource module and an expert module that are operable to store reference material and to 

provide that material to learners. The system further includes an analysis module that is operable 

to perform statistical analysis and provide results, such as charts and graphs. Next, the system 

includes a progress report module that is operable to provide a learner with information about his 
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or her answers and justifications and an interaction module that is operable to allow a learner to 
interact with other answers and justifications and with other learners. A learner interface module 
provides a connection between the learner and the system. 

[0008] In another embodiment, the present invention provides an authorable, 

modularized computer-based system that includes a features module that is operable to allow an 
author to choose the features that will be offered in the system. In addition to the modules 
mentioned above, the system of this embodiment includes a learner feedback module that is 
operable to allow the author to select the information that the system will provide to learners 
concerning the learners' answers and justifications and a data reporting module that is operable 
to allow the author to determine the information he or she will receive concerning the learners' 
answers and justifications. This system further includes an author interface that provides a 
connection between the author and the system. 

[0009] In yet another embodiment, the present invention includes a method in which an 

author selects the features a system will employ. Thereafter, the author inputs information and 
questions. The author also inputs reference material and chooses learner feedback and learner 
interaction options for those features selected. Next, the author selects progress report and data 
report options if those features are selected. 

[0010] The systems and methods of the present invention teach critical thinking skills in 

a computer-based environment, and particularly in a computer network-based environment, by 
requiring a learner to provide a justification for his or her answers concerning information 
provided. Additionally, the systems and methods of the present invention provide reference 
material for a learner to review before and after answering a question and/or providing a 
justification and allow a learner to compare his or her answer and justification to those of other 
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learners and experts. In this manner, the learner is efficiently and effectively taught critical 
thinking skills. 



BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING 
[0011] The present invention is described in detail below with reference to the attached 

drawings figures, wherein: 
[0012] FIG. 1 is a block diagram illustrating an exemplary computing system 

environment in which the present invention may be practiced; 
[0013] FIG. 2 is block diagram illustrating an exemplary network environment in which 

the present invention may be practiced; 
[0014] FIG. 3 is a block diagram of a method according to one embodiment of the 

present invention; 

[0015] FIG. 4 is a block diagram of an exemplary electronic page according to one 

embodiment of the present invention; 
[0016] FIG. 5 is a block diagram of an exemplary electronic page according to one 

embodiment of the present invention; 
[0017] FIG. 6 is a block diagram of an exemplary electronic page according to one 

embodiment of the present invention; 
[0018] FIG. 7 is a block diagram of an exemplary electronic page according to one 

embodiment of the present invention; 
[0019] FIG. 8 is a block diagram of an exemplary electronic page according to one 

embodiment of the present invention; 
[0020] FIG. 9 is a block diagram of an exemplary electronic page according to one 

embodiment of the present invention; 

4 

1271215vl 



[0021] FIG. 10 is a block diagram of a method according to one embodiment of the 

present invention; and 

[0022] FIG. 11 is a block diagram of a system according to one embodiment of the 

present invention. 

DETAILED DESCRIPTION OF THE INVENTION 
[0023] The present invention provides a system and method for use in a computing 

environment for teaching critical thinking skills. The systems and methods include providing a 
learner with information, asking the learner to answer questions regarding the information and 
then asking the learner to justify his or her answer. The systems and methods then provide the 
learner with results comparing his or her answers and justifications to other learners and with an 
expert. The present invention may also provide resource material to the learner to assist in 
answering questions and developing justifications and to aid in understanding suitable answers 
and justifications. By using the present invention a learner is taught critical thinking skills in a 
timely and optimal fashion. 
[0024] Referring now to FIG. 1, an exemplary computing system environment on which 

the invention may be implemented is generally denominated by the numeral 10. It should be 
understood that computing system environment 10 is only one example of a suitable computing 
environment and is not intended to suggest any limitation as to the scope of the invention. As 
those skilled in the art will appreciate, the invention may be practiced with other computer 
system configurations. The invention may also be practiced in distributed computing 
environments where tasks are performed by remote processing devices that are linked through a 
communications network. 
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0025] Computing system environment 10 includes a general purpose computing device 

in the form of a computer 20 that includes a processing unit 30, a system memory 40, and a 
system bus 50 that couples various system components including system memory 40 to 
processing unit 30. Computer 20 typically includes a variety of computer readable media. By 
way of example, and not limitation, computer readable media may comprise computer storage 
media and communication media. System memory 40 includes computer storage media in the 
form of volatile and/or nonvolatile memory such as read only memory (ROM) and random 
access memory (RAM). 

[0026] Computer 20 may also include other computer storage media. By way of example 

only, FIG. 1 illustrates a hard disk drive 60 that reads from or writes to nonremovable, 
nonvolatile magnetic media and an optical disk drive 70 that reads from or writes to a removable, 
nonvolatile optical disk such as a CD ROM or other optical media. Hard disk drive 60 and 
optical disk drive 70 are typically connected to system bus 50 through a non-volatile memory 
interface such as interface 80. The drives and their associated computer storage media may 
provide storage for computer readable instructions, data structures, program modules and other 
data for computer 20. 

[0027] A user may enter commands and information into computer 20 through input 

devices such as a keyboard 90 and a mouse 100. (It should be understood that mouse 100 is one 
example of a pointing device. Other pointing devices include a trackball, a touch pad and a light 
pen.) These and other input devices are often connected to the processing unit 30 through a user 
interface 110 that is coupled to system bus 50, but may be connected by other interface and bus 
structures, such as a parallel port or a universal serial bus (USB). A monitor 120 or other type of 
display device is also connected to system bus 50 via an interface, such as a video interface 130. 
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In addition to a monitor, computers may also include other peripheral output devices such as 
speakers and a printer (not shown). 

[0028] As will be explained further below, in the present invention computer 20 may 

operate in a networked environment using logical connections to one or more remote computers, 
such as a remote computer 140. These logical connections may include a local area network 
(LAN) and/or a wide area network (WAN), and may also include other networks. In addition, it 
should be understood that portions of these networks can be wired, wireless, or some 
combination thereof. When used in a networking environment, computer 20 may be connected 
to the network through a network interface or adapter 150. In addition, computer 20 may include 
a modem or other means for establishing communications over a network. In a networked 
environment, program modules or portions thereof may be stored in the remote computer 140. 

[0029] FIG. 2 illustrates an exemplary computer network environment on which the 

invention may be implemented. It should be understood that the computer network environment 
shown is only one example of a suitable network environment and is not intended to suggest any 
limitation as to the scope of the invention. As those skilled in the art will appreciate, the 
invention may be practiced with other network configurations. 

[0030] As shown in FIG. 2, web server 160 is interconnected to a number of other server 

computers, such as a database server 170, a file server 180 and a mail server 190. Web server 
160 includes a document storage device 165. Similarly, database server 170, file server 180 and 
mail server 190 include document storage devices 175, 185 and 195 respectively. In this 
example, web server 160, database server 170, file server 180 and mail server 190 are part of a 
local area network 200. A wide area communications network 210 (e.g. the Internet) permits 
remote Web sites 220 and client computers 230, 240 and 250 (each equipped with a browser 
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such as browser 235) to gain access to web server 160, for example to search for documents or 
other forms of electronically stored information. 

[0031] Generally, a client computer, such as computer 230, includes a browser function 

or separate browser application 235 that may locate and display documents to a user. When a 
user at client computer 230 desires to see a document, browser 235 causes client computer 230 to 
issue a request that includes the URL (Uniform Resource Locator) value for the document sought 
in an HTTP (Hypertext Transfer Protocol)-coded command. A URL value is a unique address 
which fully specifies the location of a content object on the Internet. HTTP is the standard 
World Wide Web client-server protocol used for the exchange of information (such as HTML 
documents, and client requests for such documents) between a Web browser and a Web server. 
HTTP includes several different types of messages that can be sent from the client to the server 
to request different types of server actions. 

[0032] After receiving such a request, Web server 160 will either obtain the HTML 

(HyperText Markup Language) page, for example from file server 180, or assemble the HTML 
page from an HTML file. HTML is a standard coding convention and set of codes for attaching 
presentation and linking attributes to informational content within documents. During a 
document authoring stage, the HTML codes are embedded within the informational content of 
the document. When the Web document is subsequently transmitted by a Web server to a Web 
browser, the codes are interpreted by the browser and used to parse and display the document or 
page. An HTML document is a special type of document which includes HTML codes to permit 
the document to be viewed using a Web browser program. An HTML document that is 
accessible on a World Wide Web site is commonly referred to as a "Web document" or "Web 
page." 
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[0033] If the HTML page is assembled, Web server 160 will process the information 

stored in the HTML file, extract any queries from the file and issue those queries, for example to 
database server 170. Database server 170, in turn, may issue queries to file server 180 or some 
other computer. Web server 160 will receive the results of these queries and merge that 
information with the contents of the HTML file to generate an HTML page. The HTML page is 
then routed via communications network 210 to client computer 230 where browser 235 operates 
to display the page. The operation of Web browsers, the Internet, the World Wide Web and 
HTML authoring systems are well-known in the art. 

[0034] As stated above, the present invention may be embodied as a method or a system 

implemented in a computing environment. Thus, when implemented, the present invention will 
be composed of computer-executable instructions, such as HTML instructions, that may be 
grouped together in program modules. Generally, a program module will include programs, 
objects, components, data structures, etc. that perform particular tasks or implement particular 
abstract data types. In a distributed computing environment or a computer network environment, 
program modules may be located in one or more local and/or remote computer storage media 
including memory storage devices. Additionally, if the method is implemented on a server, such 
as, for example, web server 160, then the learner may be present at a remote computer, such as 
client computer 230, and may access the server over a communications network, such as the 
Internet. 

[0035] Referring now to FIG. 3, in one embodiment the present invention is a method 

implemented in a computing environment. According to this embodiment of the present 
invention, the method begins in box 260 where informational content and a question or questions 
are presented to a learner. The content may be in any format available for use with a computer 
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including, but not limited to, text, pictures, video, audio, or any combination thereof. The 
content and question may be presented in the form of an electronic page or document, such as a 
Web page, that is capable of being displayed by, for example, a browser function or application. 
[0036] From box 260 the learner may request assistance in the form of a "coach" and 

thereby move to box 270. It should be understood that in some embodiments, "coaching" may 
be mandatory and, therefore, the method will move to box 270 when, not if, the learner selects 
the coach. In box 270, "coaching" is provided to the learner. Coaching includes providing 
information in the form of text, audio and or video in advance of or in conjunction with a learner 
answering a question and/or providing a justification in order to assist the learner in making an 
informed decision and forming higher-level rationales. After providing the coaching, the method 
returns to box 260. 

[0037] From box 260 the learner may request additional reference or resource material 

related to the subject matter at issue and thereby move to box 280. In box 280, resources that the 
learner may utilize, for example, to understand the question presented in box 260 are provided to 
the learner. The resources may include, but are not limited to, titles, excerpts or summaries of 
books or articles that the learner may obtain or links to pertinent Web pages. After providing the 
resource material, the method returns to box 260. 

[0038] After reviewing the content and question, the learner must provide an answer to 

the question and a justification for his or her answer. When the learner provides an answer, the 
method moves to box 290 where the answer is received and stored. When the learner provides a 
justification, the method moves to box 300 where the justification is received and stored. After 
receiving an answer or a justification, the method returns to box 260. When ready to proceed, 
the learner will submit his or her answer and justification, and the method will move to box 310. 
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[0039] In box 310, an analysis is performed. This analysis may include, for example, 

entering the learner's answer and justification for the particular question into the population of 
all learner answers and justifications for that question and then determining the average answer, 
the distribution of answers, or other statistical measures. The analysis may also include forming 
visual aids such as a bar graph or pie chart that illustrates the distribution of the answers. 

[0040] Once the analysis is performed, the method continues in box 320 where results are 

displayed for the learner. The results may include, for example, the visual aids formed in box 
310. The results may also include other learner's actual answers and justifications. 

[0041] At box 320, the learner may request an expert judgment and thereby move the 

method to box 330. At box 330, an expert judgment is provided the learner for his or her review. 
This judgment is much like the coaching discussed previously in that it may be in the form of 
text, audio or video. Unlike the coaching information, however, the expert judgment provides 
specific information about an answer and justification provided by an expert in the field. 
Following the learner's review of the expert judgment, the method returns to box 320. 

[0042] Finally, at box 320, the learner may elect to interact with other learners and/or the 

other learners' answers and justifications and thereby move the method to box 340. At box 340, 
the learner interacts with other learners and/or their answers and justification. This interaction 
may include, but is not limited to, reading other answers and justifications, ranking the 
justifications, rating the best justification, and e-mailing a comment about a justification to 
another learner. 

[0043] To further illustrate the method discussed above, the following example is 

provided. It should be understood that this example is provided for illustrative purposes only and 
is not intended to limit the scope of this invention. In the example, the present invention is 
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utilized to instruct learners about the qualities teachers consider in student writing and to develop 
the learner's ability to assess student writing. The critical thinking aspect of this example is the 
development of the learner's ability to assess student writing. Learners in this example may 
include, for example, actual teachers and student-teachers. 

[0044] Referring first to FIG. 4, the invention initially provides the learner with an 

electronic page 350 containing a student writing sample 360 and a question 370 that asks the 
learner to provide a numerical rank for the writing sample based on certain criteria, such as 
sentence structure or word choice. The page may present writing sample 360 as a typed 
document or as a "snapshot" of the original paper, which would show the writing in the student's 
handwriting. Page 350 may also contain additional information that explains each rank in terms 
of the criteria. As shown in FIG. 4, for example, a rank of 1 is indicative of a beginner while a 
rank of 4 is indicative of an advanced writer. The additional information could provide that a 
beginner is a writer that is "not yet showing control" and that an advanced writer "exceeds 
expectations." Page 350 also contains an area 380 for entering a justification, a coach button 
390, a resources button 400 and a submit button 410. 

[0045] Referring now to FIG. 5, if the learner selects coach button 390 on page 350, the 

invention provides page 420 containing a picture 430 of an experienced teacher, a brief 
biography of the teacher 440 to establish the teacher's credibility, and a quote 450 from the 
teacher that offers more detail about the criteria in question and the rankings. The learner may 
read the quote or select audio button 460 to listen to a recording of the quote. Closing page 420 
returns the learner to page 350 in FIG. 4. 

[0046] Referring now to FIG. 6, if the learner selects resources button 400 on page 350, 

the invention provides page 470 which contains an excerpt from a teacher handbook and titles 
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for other scholastic references the learner may obtain to learn more about the criteria. Closing 
page 470 returns the learner to page 350 in FIG. 4. 

[0047] After the learner enters a rank and a justification and selects submit button 410 on 

page 350, the invention saves the responses in a database and calculates the average ranking 
among all saved rankings in the database for the particular question and prepares a bar graph 
showing the distribution of the rankings. The invention then provides electronic page 480, as 
shown in FIG. 7, with the average ranking, the distribution bar graph, and an expert ranking 
along with the learner's ranking and justification. This page also contains an expert button 490 
and an interaction button 500. 

[0048] Referring now to FIG. 8, if the learner selects expert button 490 on page 480, the 

invention provides electronic page 510 containing a picture 520 of an experienced teacher, a 
short biography 530 of the expert, and a quote 540 from that teacher that states the expert's 
ranking and justification. The learner may read the quote or select audio button 550 to listen to a 
recording of the quote. Closing page 510 returns the learner to page 480 in FIG. 7. 

[0049] Referring now to FIG. 9, if the learner selects interaction button 500 on page 480, 

the invention provides page 560 containing other learners' rankings and justifications for the 
learner to read and an area to enter an e-mail comment about a particular justification. Closing 
page 560 returns the learner to page 480 in FIG. 7. 

[0050] In FIG. 10, in yet another embodiment, the present invention is a method in which 

an author is able to create a customized version of the invention. The method begins at box 570 
when an author selects or chooses the features a system will employ. In this step, for example, 
the author will decide whether to provide a coach function or a resource function and whether to 
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make consultation of these functions mandatory. It should be understood that if the author 
chooses to omit a particular function, then the method will skip the box addressing that function. 
[0051] The method then continues at box 580 where the author inputs the informational 

content. As indicated above, the informational content may be in the form of text, audio, video 
or some combination thereof. Next, at box 590, the author inputs the queries that are tied to the 
informational content. 

[0052] At box 600, the author will input the coaching information. In the example 

above, "coaching information" included a photograph, a biography and a quote. It should be 
understood that "coaching information" is not limited to these types of information. 

[0053] At box 610, the author chooses the learner feedback options, or those parts of the 

learner's responses that will be displayed. For example, the author may choose to allow the 
learner to view only his or her own answer and justification or the author may elect to allow the 
learner to view graphical and/or numerical feedback on group performance and a running 
average of the learner's ratings along with a discrepancy score for their rating and the expert's 
ratings. 

[0054] At box 620, the author inputs the expert information. As with the coaching 

information and the informational content, the expert information may be in any computer 
readable format. It should also be noted that "expert" answers and justifications also may be 
seeded into the learner answer and justification storage so that when a learner reviews other 
learner assessments (assuming the author has authorized this conduct) he or she will also review 
the expert assessments. 

[0055] At box 630, the author chooses the learner interaction options. For example, the 

author may allow the learners to review their peers answers and justifications, rate their peers 
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answers and justifications and e-mail comments to other learners. Additional types of interaction 
may also be chosen if supported by the computing environment such as, for example, entering 
into a real-time dialog with other learners currently using the customized system or monitoring a 
dialog between other users, which may include experts. 

[0056] In box 640, the author chooses the progress report options. In this step, the author 

determines the measures of knowledge, motivation and interest that will be made available to 
learners, such as measures of self-efficacy and/or queries of learner likes and learning during 
their work within the application, and how progress will be reported to learners (e.g. by scoring 
and reporting test scores immediately or by reporting progress through a site map). 

[0057] In box 650, the author chooses the data reports that will be made available to the 

author and /or researchers. For example, the author may elect to generate reports concerning 
learner demographics, or concerning site performance (which may include timing — how long 
between content presentation and submission of assessment? - and choice information), or 
concerning measures of learning and motivational outcomes. Additional possibilities for data 
reporting include, but are not limited to, number of cycles per learner, number of uses of the 
Coach or Expert feature, and correlations between use of the Coach feature and discrepancy 
between the learner and Expert assessment. 

[0058] Continuing on to FIG. 11, in another embodiment, the present invention provides 

an authorable, modularized computer-based system generally referred to by the numeral 660. 
Initially, system 660 includes a content module 670 and a query module 680 that are operable to 
receive and store informational content and questions respectively and to provide the information 
and questions to learners. In addition, query module 680 is operable to receive and store learner 
answers and justifications. System 660 also includes a coach module 690, a resource module 
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700 and an expert module 710 that are operable to store reference material and to provide that 
material to learners. It should be understood that, in addition to a custom-written computer 
program, many commercial database applications may serve as content module 670, query 
module 680, coach module 690, a resource module 700 and an expert module 710. 
)059] System 660 includes an analysis module 720 that is operable to perform statistical 

analysis on information, such as learner ratings associated with particular questions, and provide 
results, such as charts and graphs. As with the modules discussed above, it should be understood 
that many commercial applications, such as a spreadsheet application, may serve as analysis 
module 720. 

3060] Continuing with FIG. 11, system 660 includes a progress report module 730 that 

is operable to provide a learner with information about his or her answers and justifications and 
an interaction module 740 that is operable to allow a learner to interact with other answers and 
justifications and with other learners. Next, system 660 includes a learner feedback module 750 
that is operable to allow the author to select the information that the system will provide to 
learners concerning the learners' answers and justifications, a data reporting module 760 that is 
operable to allow the author to determine the information he or she will receive concerning the 
learners' answers and justifications and a features module 770 that is operable to allow an author 
to choose the features that will be offered in the system as discussed above. 

3061] Finally, system 660 includes a learner interface module 780, which provides a 

connection between a learner workstation 790 and system 660, and an author interface 800, 
which provides a connection between an author workstation 810 and system 660. Interfaces 780 
and 800 may be graphical user interfaces. 
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3062] From the foregoing it will be seen that this invention is one well adapted to attain 

all the ends and objects set for above, together with other advantages, which are obvious and 
inherent to the system and method. It will be understood that certain features and sub- 
combinations are of utility and may be employed without reference to other features and sub- 
combinations. This is contemplated and within the scope of the appended claims. Moreover, 
while particular embodiments of the invention have been illustrated and described in detail 
herein, it should be understood that various changes and modifications might be made to the 
invention without departing from the scope and intent of the invention. The embodiments 
described herein are intended in all respects to be illustrative rather than restrictive. Alternate 
embodiments will become apparent to those skilled in the art to which the present invention 
pertains without departing from its scope. 
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